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ABSTRACT 

The evaluation report is one of seven produced for 
the Occupational Exploration Prograa (OEP) , a series of simulated 
occupational experiences designed for junior high school students. 
Describing the pilot testing of the simulation dealing vith 
education, the report contains sections describing the simulation 
context, evaluation procedures, results^ and a Beviser*s Information 
Summary (RIS) • In the simulation students worked on the design and 
development of a school for the Spaceship Scorpio, about to take a 
long voyage in space. The occupational roles simulated included 
educational researchers, consultants, community members, and other 
student-selected educational roles. The experimental design involved 
two Colorado schools, with a total of four experimental and four 
control groups involving 97 eighth and ninth graders. Instrumentation 
included knowledge and affective testing, student and teacher 
questionnaires, and a panel review. Analysis of variance and other 
descriptive statistics were employed and reliability estimates were 
calculated. Analysis of variance results revealed that the simulation 
had a positive impact on student occupational knowledge, but no 
impact on student occupational preferences. The RIS records and 
extrapolates trends related to the strengths, weaknesses, and 
recomnendations from all data sources. Appended materials include the 
evaluation instrunents used and the teacher log. (MN) 
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ABSTRACT 
PLAin-miG EDUCATIOITAL FRGGRAIg 
g/ALUATION REPORT FOR TitS OCCUPATTaMi EXPLORATION PROGR/C>I 
By: Janes V7. Altschuld; Sandra Pritz * 



This report is onn of seven emluation reports produced for tlie Occupational 
Exploratio:i Prograr.. The Occraational i;:q>loratior. Program (O.S.P.) is 
funded by the national Institute of Edxication and is a ^'oint development 
effort of The Center for Vocational Education (The Ohio State University) 
and the Jefferson Covmty, Co.lorado puhlic schools, O.E.P* is a series of 
experiences designed to provide junior high school students \nth the oppor- 
tunity to exnlore occunations. One of the major vehicles for e:q)loration 
is the simulation technique- In •PY' 197^^, 12 simulations vrere developed 
and seven of those ti.^lve were pilot tested. This report describes the 
pilot testing of the simulation dealing v/ith education. The report contains 
sections describing simulation context, evaluation procedures, results and 
a Eevisor's Information Sunssary (RIS)- The KIS is useful for a variety of 
piarposes and includes the stren^chs of the simulation as well as its 
weeknesses, Beloi-r is a synopsis of the specific content of the report. 

SBIDIATION CCIigSiCT : In this siiTiulation^ students work on the de&ign and 
development of a school for the Spaceship Scorpio. Students are told that 
Scorpio is going to be on a Ions voyage into space. The ship i/ill not only 
have crew members aboard but due to the length of the voyage, families of 
the crew mil also come along. The problem situation confronting st\idents 
then is to develop or design a school for the spaceship. In this simulation, 
the occupational roles include educational researchers, educational consul- 
tents, coismunity members and other educational roles which students select 
for the spaceship's school. S^ISRSSHTAL DESIGN : For evaluating this 
simulation, tvro schools, one in Jefrerson County, Colorado and one in 
Denver, Colorado were used, each school ha\d.ng experimental and Uto 
control groxips. A teacher facilitated the implementation of the simulation 
irLth each experir.ental group. The experimental and control groups consisted 
of 8th and 9th graders; the four experimental groups totaled 3o students 
and the four control gi^oups totaled 39 stud?: ':j. :L;:? r/.TIOII : 
A lt5 item multiple choice kno^rledge test, '"^/hat Lo You Knov;?'', and as item 
affective test, 'Vnat Do You Like?", were administered as pre- and posttests 
measuring student occupational, knowledge gain and attitudinal change. The 
student post ::odule questionnaire* ''I'Jhat Do You Thinlc?'*, administered to 
the experir^cntal group after completion of the simulation, measured student 
perceptions of the module. Teacher questionnaires and a panel ve\-ie\T were 
designed for the purpose of obtaining teacher perceptions of the simu''ation. 
AI-L\LYSI S: The knov-rledge test and affective test results were derived 
thi^ouch^ana^ses of variance. Other descriptive statistics were employed 
where appropriate (i.e., frequency, percentage). Reliability estimates were 
calculated -co obtain the internal consistency estimates of the knovrledge 
test and to detei-mine inter-coder and intra-coder agreement for the attitude 



BEST COPY AVAILABLE 



scale. RI^SULTS : Tr.o A\X-VA '^'•:'•»alt3 rovcn.1 that ^ho simulation hr^d a 
positive iri^pact on ctudcnt or^.c ^.pation/.l ICiio^iledGC in the education field 
(p<,C::): hut, no apparent chcn^os in student occu^vitional preferences 
ve^:. ::o: 'd. Cor/ va.l:.y, the j;inulation \'as perceived to "be successful from 
stxidcnfc c.nd teaclv.r c*03aents collected fro:.a questiomiaire data. REVISORY S 
SnrOR^ lM IO:! Sl^'i:-L V'(Y: The RIG v;as desiiyied to not only assist revisers to 
assiiziioxe inforr!:\rion oo21ect-?d durin;*; the pilot-test, but also as a unique 
vay of siurr>arizing Ihe dat?,. T*\e J:urcr^ary is a record of the strengths, 
weaknesses and recorruendatio-"..- i*or re*. isors froni all data soui'ces (i.e., 
student tests, student questio:\naires , teacher questionnaires, etc,)* 
Trends have teen eirbrapolated vhich list the most apparent strengths and 
weaknesses of the sinnilation as ^.^311 as reccsmendations to be considered 
in the revision of the sinulation. 
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I. Brief DeBcriptlon of the Module 

The education mcxlule consists of a preview, a preparation section, 
five major tasks or activities and a siuHnary section. The Prevlev is a 
slide tape presentation designed to aiotivate the student as well as to 
create a mentail set for poxtici^ting in the xnodule. The presentation 
continually depicts man's primitive past in juxtaposition to or inter- 
spersed with pictures of recent space achievements. The student is told 
that in this simulation he/she wl3JL be woorking on designing or developing 
a school for a spaceship. The preview is scheduled for about a period. 

The Prepar-ation section is a slide-tape presentation describing 
Spaceship Scorpio. After viewing the presentation students examine a 
diagram of the spaceship that is included in the Preparation Handbook 
Resource Packet. For the I^eparation, students are told that Scorpio is 
going to be on a long voyage into space. The ship would not only have 
crew members aboard but, due t-^ the length o:f the vpyage, families of the 
crew would also come along. The problem situation confronting students 
then is oc develop or design a school for the spaxieship. The preparation 
section is scheduled to last appraxiinately one period* 

Handbook 1 > Identifying People and Courses deals with describing 
the Scorpio community and determining the educational needs of the com- 
munity. Students are divided into two groups, educational researchers and 
educational consultants. The groups conduct a mock suanr^ of the Scorpio 
caaminlty— the researchers focus on the age, sex, occupation, etc., of the 
Scorpio space family, and the consultants study educational needs, hobbiei, 
etc., of the population. As a result of the surv»5y, the curriculum^ for 
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the ScQorpio Space School is decided upon. Roles in this task include the 
educational consultant ^ the educational researcher, cmd coonmity members. 
The suzrey and designing the curriculum occupy ahoot two or three periods • 

In Handbook 2 > Deciding What Staff is Needed ^ the students form a 
planning condttee for the purpose of determining idiat staff is needed in 
the school* Ihe students study careers now existing in the schools to 
see nhat particular staff members th^ vould like to include in their 
school* Due to the limited size of the ship, only ten educators or 
individuals in educati<xi-related careers will be alloired to paartlcipate 
In the voyage* Roles in this task include only membership In the 
p1ann5.ag comnlttee of the school. Time required is about one period. 

In Handbcok 3 » Hiring the Staff s the students are required to fill 
out applications for the ten educational roles and to then Intervleir for 
the position of their choice. At the end of the task students are in 
roles and ready to begin planning the school* s facilities and supplies. 
Roles here include being members of the Intervievlng ccomittee and roles 
in education as designated or chosen by students. 

Folloving upon the experiences of Handbook 3^ the students in Task k 
are re<iuired to Plan Facilities and Supplies . To accosgiLish this they 
vork both individually and in small groups to lay out the physical design 
of the Space School and to determine the stqpplies that iflll be needisd. 
Roles in this task are the sane as those chosen by the students Jii Task 3. 
Task 3 and k require about three to four and one to two periods^ 
respecti^ly. lu Task 5> Evaluating the School Rrogram > students conpare 
the school and school program that they have designed against the survey 
results that th«y tabulated and analyzed in Task 1. Again roles are 
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Identical to those chosen bj students In TMk 3. Suggested tins for 
Task 5 It ooe clus parlod» 

The Sunnaiy Hiaae conalsts of the studenta conducting a "Tonorroir 
Shon** In irtd.ch they assuicd the rde of space travelers returning trom, an 
extended space voyage. Th«y relate to others (students, teasers, etc.) 
their perceptions of the educati<mal irork they did nhile aboard the ship^ 
Roles are essentially identical to those in Tasks 3, k and Thi& task 
is scnewhat flexible and nsy take as losny as two class periods. 

The overall length of this siaulation vas to be apprcxiMitely 
fifteen periods. Aside from the roles of the educational consultants 
and researchers ^ the other educational roles vere flexible depending upon 
the individual interests of students and their perceptions of what 
constitutes a school for spax:e. 

II. Description of Evaluation Procedures Biployed 
A. Specific Saagle Used ^ 

1. Schools • for this nodula cme Jefferson County and one 
Denver school were used. In each sdxool there were two 
experimental and two control groups « The schools and the 
teachers were selected via discussion with adsdnistrators 
and teachers in each of the districts. A brief description* 
of the schools follows. 

Alaneda junior High School (c&rades 7-9) > Jefferson Couaty » 
Alaaeda Junior High School is a ssiall school with 
approodaately 700 students in grades seven thrcui^ nine. 

^Descriptions were obtained by John Badloff of the Jefferson County project 
staff. 




It seemfi to have a fairly stable school population in 
that school Mcords indicate that over seventy 
percent (72^) of the ninth grade population have been 
in this particular school for three consecutive years* 
AdditiOQaUy, very few of the ninth graders have attended 
wore than two elemntary schools* LorgeJISioniUke tests 
adadnistered at the school indicate a fklrly normal 
distribution student ability* The distribution of 
parental occupations shorn that kQ percent of the Mothers 
are working and that almost 3^ percent of the fathers are 
in managerial, professional or skilled positions* The 
school population is primarily Caucasian (93 percent) with 
the remaining seven percent coming trcm other minDrit^ groups* 
Hamilton JUnlor High School (C&rades 7--9)» Denver* 
Hamilton Jtinior school is a large school with approK* 
imately 1,600 students enrolled in grades seven through nine* 
The area served by the school is quite large and over sixty 
percent of the students at Hamilton are bussed in each day* 
The students generally cone frm the middle income range but 
there are some students txm, yxpper income areas* Student 
au^hievement Bttms to be relatively high« (According to the 
assistant principal, over half of the seventh grade students 
maintain a B or higher academic average* ) Ibe racial make- 
up of the school is estimated to be 80 percent Caucasian and 
about 20 percent in minority gtc^xpa^ Further specification 
of the pcfpulatlon was not available for this school* 
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2. sample vlthln Schools 
a. Teachers 

In Alaaada Jimlor Hl^ School one faflMOe aoA one aale 
teacher volunteered to participate in the teaching of 
the module. The folloving demosraphic data ea the 
teachers was collected: 
Years of Teaching Eacperlence 

1 year (N»l), 2 to U years (ll«l) 
Subject Area Usually Taught 

English (»-2) 
BriLor Eacperience vith Simalation Technitnea 

As an observer (lf=:l) 

■o data ayailable (K-1) 



In Haoilton Jxmior two male teachers volunteered to 
participate in the teaching of the module. The 
folloifing demographic data on th.i teachers vas 
collected: 

Years of teaching eacperience 

2 to h yews (N==l), 6 to 8 years (N«l) 
Subject area usually taught 

Social Studies (N^2) 
Rrior experience vith simalation techniques 

As both a teacher and perticipant (11*1) 

No prior eacperience (N-l) 
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b. Stttdeats 

The nineteen students troBk Haadltcm Junior Hic^ School 
(Denver) i«fao participated in the module imre volunteers 
from physical educaticm classes* The overall balance vith 
regard to sex was ei^t aales anl eleven femles in the 
ezperlnental classes. The students vere all eii^xth graders. 

The seventeen students tram, Alaaeda Junior Hi|^ (Jef- 
ferson County) ufao participated in the module were mostly 
Tolunteers from eigjtth grade language arts classes. The 
overall balance vith regard to sex vas six males and 
eleven females. 

The cOTtrol grcups in Denver irere students seXected 
troBx other than regular classroom settings. For analysis 
purposes and to acccoBodate the need to have tvo ccaqparable 
ccMitrol groups « the students were randomly assigned to 
either Denver Control Oroop #1 or Control Qtcfop ^jZ. The 
total number of students involved in the control grciqMi 
is tventy-tvo vith the male»female balance being twelve and 
ten, respectively. 

The contxrol graqp in Jefferson County consisted of 
the two eighth grade language arts classes. The number 
of students involved is thirtynieven irith the male to 
female balance heing fifteen and twentj^two, respectively. 

In suomry, the sanpling vas far from ideal. It 
vas impossible to conduct more systematic saq^ng due 



ERIC 



m 

13 



7 



to program and organizational constraints within 
buildings* As will be described later in the 
experimental design results section, initial pre- 
test group differences were present. 

It also should be noted that experimental 
results are based only on students who took both 
the pre and posttest. There was sample loss in the 
testing of the module as follows: 

- Denver experimental groups, all nineteen »tu- 
dents who started the eiqperiiiental program 

completed it (zero percent loss); 

- Jeff Co experimental groups , two of the 
nineteen students who started the progrcun 

did not complete the posttest (eleven percent loss); 

- Denver control groups > nine of the thirty-one 
students who took the pretest, did not com- 
plete the posttest (twenty-nine percent loss); 

- Jeff Co control groups , six of the forty-three 
students who took the pretest, did not complete 
the posttest (thirteen percent loss). 
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Saisple loss is always difficult to account for 
in an experii&ental situation* Sone students may have 
been sick or otherwise out of the classroom during the 
pre or posttesting tine* The logistical set-up for 
the test of this module required that an administrator 
be present at each testing session* Provisions for 
folloir«up testing of students who missed a session 
were not feasible given the available waxtpaw& in the 
field. Some students may simply have avoided taking 
the tests. The sample loss in this instance does not 
seem to be large enou^^ to inv6^.idate the results of 
the experimental design and efforts will not be made 
to study the loss in any detail. . 

B. Types of Classes and Groupings 

Knowledge of the type of class or group setting in which the module 
has been tried is importeuit information in regard to intexTpretlng the 
module results. In Denver > the students who participated in the module 
were able to experience it in a manner close to that intended by developers. 
Small (N=9 and 10) groups of student volunteers were used in the pilot 
test* They used the module independently of or without the distrac- 
tion of other groups of students* No doubling up on roles was necessary* 

In Jefferson County, a similar situation prevailed* Students, for 
the most part, volunteered to participate in the simulation* Two small 
groups were thus formed* Tho groups used the module independently of 
the rest of the class* In addition, they were kept physically separ- 
ated from other students in the class* 




15 



9 



C. Experimental Design as laplenented 

Glrea the saall size of tUe experliient&l groups it mui decided 
not to partition the design Ify sex as specified in the proposal. 
This eliminates the possibility of studying the test scores of males 
and females. Aside fron this one smauLl change , the design is basically 
that stated in the proposal. Schematically the design could be 
depicted as belov: 



Figure 1 - Scheiaatic of the Bxperimeatal Design 
Foot The Education Module 



Hamilton 
(Denver) 



Alameda 

(Jefferson 
County) 
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Pottttst 


Bxperiuental Group #1 




1 

h 


£xperiaeat&l Group 42 






Control Group #1 


h 

1 
1 
1 


h 

1 

1 
1 


Control Group #2 






ExperlAental Group #1 






Experimental Group #2 






Control Group #1 






Control Groap #2 







*In order for a student's scoi'es to be included in the analysis, he/she would 
have had to participate in both the pre and posttest. 
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The analysis vlll be the sane aa designated In the paroject 
proposal for the Occiqpational Exploration Program (TY^lk) vlth the 
exception that the sex varli^le haa been deleted. Of key Interest 
will be the interaction between the experimental^control variable 
and the pre and postteat variable. If the aodule has had an iapact 
upon students^ a significant interaction would be escpected with the 
source of the interaction being a sizeable experimental groi:^ gain 
on the posttest. Separate analyses will be run for the total 
cognitive test scores as well as for one diiaenslon of the attitud- 
inal scale. The analyses will be in accordance with the abbreviated 
suanary table shown below. 
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Table 1 - Peurtlal Anava Susmary Table 
For The Education Module 





Source* 


df 


Potential F Test 




Between Students 


abcn-1 




Term No. Betveen Classes 






1 


A 


a-l 




2 


B 


b-1 


2/k 


3 


AB 


(a-l)(b-l) 


3A 


k 


C/AB 


ab (c-l) 


V5** 




Wlthjn Classes 


abc (n-1) 




5 


B/C/AB 


abc (n-1) 






Within Students 


abcn (d-l) 




6 


D 


(d-1) 


6/10 


7 


AD 


(a-l)(d-l) 


7/10 


8 


BD 


(b.l)(d-l) 


8/10 


9 


abd) 


(a-lj(b-l)(d-l) 


9/10 


10 


cd/ab 


ab (c-l(d-l) 


10/11** 


n 


ed/c/ab 


abc (d-l) (n-1) 






Total 


abcdn-1 





*A brief discussion of the variables will be Included in the text innediately 
foUoidng this table. 

^HiThe results ftcm the two stcurred F tests are lioportant in that if 

the test yields an Insignlglcant F ratlo^ then the two terms k and 3f ^nd 10 
and 11^ could be respectively pooled and used tor the reme^lnder of the 
appropriate F tests. 
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The Independent variables for thit inodule are described b^lcw: 



Variable Description 

A Treatiaent (experimental vs* control) 



B 



Schools (Denver vs. Jefferson County) 



Classroonus (N«d) 



Testiii-5 (Rre. vs» Post) 



Students 



Fixed; betveen lervele 
of C 

Fixed; between levela 
of C 

Randon; nested within 
AB 

Fixed; within S's 
(repeated measure) 

Random; nested within 
ABC 
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D» Inttrunentatlon ^ inatrutteat Spectflca 

1. Khonfladge Test - What Do Yw Khoir? (The test Is appended 
to this report) 

The knoifledge test for education consisted of 45 questions. The 
test included 16 multiple choice questions; 6 True and False questions; 
a jsatchlng question with 10 psrts; a question containing a situation 
folloNdd by 7 statenents related to the situation (the student had to 
check as naoy statements cui he felt would be helpful to the situation); 
and a 6 part questlcm ifhich required the student to arrange the 6 
j/BXtB in terns of ord/?T. For purposes of anflOysis, each part of a 
question with nore than one part was treated as thouj^ it were a 
single question. Thus, the test contained k5 total questions. 

In general, the questions were at a low comrehension level in 
relation to the Bloom Taxononor. Tiro basic thrusts or areas were 
en^phasized in the tests - process and responsibility. An ezaaple of 
a process question is as foUovs: 
Test Question #10 

When evaluating a plan for an educational progras, the siost 

inqxxrtant concern should be: 

*a. Whether the plan serves the needs of the coanunlty 

b. Whether the plan provides for new textbooks every 
five /ears 

c. Whether the plan includes the teaching of world 
geography at grade 8 or grade 9 

d. Whether the plan calls for the involvenent of a 
parent-teacher organization. 

Process questions generally deal with understanding the nature 

of steps involved in doing work such as planning an educational 
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program^ understanding vhat informtion adght be necessary to perform 
a certain function ^ etc. 

Responsibility questions , on the other hand, deal generally vith 
tiho or ifhat groirp has responsibility for getting a certain Job done, 
or vho has responsibility for making decisions at a certain point in 
tine, etc. An example of a responsibility question is given beloir. 
Tes^, Question #2$ 

Which of the following groups is responsibile for pperati^ig a 

school? 

a. B&rent-Teacher Association 
*b. Board of Education 
c. National Education Association 
<i. All of the above 

2. Affective Test > What Do You Lite? (The Test io Appended To This Report) 

The affective test was designed to laeasure attitadinal change on 

the part of the student. The first five questions consist of asking 

the student if he/she wcwld like to try doing an activity. The 

student could respoz^ in one of fon^r ^ys to the item. 

-Yes, I would like to try this 
-No, I would not lite to try this 
-I*m uncertain about trying this 

-I don^t have enough information to know if I would lite to try 
this. 

The scale is scored so that the stronger the preference for trying 
to do an activity, the higher the score. Thus, yes and no responses 
receive the same sccULe value of 3, uncertain responses receive a 2 and 
not enough informatic»i types of responses receive a value of 1. These 
values are then stmsed and used in the azialysis of variance described 
earlier. Sunmed scores can vary trcm zero (no response idiatsoever) 

^Indicates correct answer 
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to 15. Note th<j scale is scored so that strength of preference, 

rather than direction of preference is the important factor (i.e., 

yes and no responses while being in opposite directions, represent the saine 

strength of preference and therefore receive the some score). 

In addition to the scaJLed responses, students were encouraged 
to state reasons for their preferences. These reasons were 
classified and in conjunction with the scaled responses, were coded 
and trcuisferred to machine scorable forms. Inter-rater and intra- 
rater agreement checks were made on the scoring process (See results 
section). 

There were 8 other questions included in the What Do You like? 
sc6LLe. The 5 questions following those just described were similar 
in nature to the earlier ones, but they required the student to 
supply or fill in the type or activity that he preferred or did not 
prefer to do. Given the totally open-ended nature of this set of 
questions, it was difficult to develop an adequate and relatively 
exhaustive scoring scheme. In addition, given the state of develop- 
ment and the early trial nature of the pilot tost, a decision was 
made to exclude results from these questions in the analysis of 
variance. 

( The last 3 questions on the test were open-ended and asked the 

students about the experiences one should have before deciding on a 

Job, the types of things that one should r^onsider before t'&king a job^ 

etc. The responses were classified and scared. Due to some difficulties 

in scoring tnese questions results will not be presented.) 

3. Student Post Modul e Quest ionnaire ^ V T nat Do You Thin k? (The 
Questionnaire is Appended To This Report) 

This questionnaire was administered to students after they had 
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coopleted the modiile and the aodulu posttest. This InBtruaent ims 
administered ooJy to the students vho participated In the module • 
!rhe content of the questionnaire relates directly to student percep* 
tlons of the module. The first tirenty questions sm in a scaled 
fomat. Questions In this set relate to a student* s inception of 
the clailty of directions, the extent to nhich the nodule Interested 
hij0iev, etc. For analysis and use, the results will be grouped and 
descriptively reported by the subject area to uhlch they pertain* 
Other questions in the qtiestionnadre deal vith parts of the siodule 
the student liked best, par^s he/she liked least, role(s) played in 
the siaulatlon, etc. These questions will be descriptlTaiy r^wsrized 
and Included on the Reviser's Information Sunnary (RIS). 
k. Teacher Evaluation Log - The Teacher Evaluation Log consists of 
five instruments packaged in one booklet and an additional instrumsnt 
to be used after the iD0du3.e was conpleted* The sixth instrunient is 
entitled ^'Genereil Module Evaluation." The instruaent order irithin the 
log parallels the ordering of the nodule* In other wards, after stu* 
dents had coopleted the Introduction to Sinulation, teachers ifould fill 
in the questionnaire regarding that part of the nodule. After students 
had coopleted the preview, teachers would fill in the questionnaire 
pertaining to the preview, and so on* Below is an instruaent by 
instrument description of the five lns\;runents contained in the log* 




17 



LISTING AND DESCRIPTICW OF TEACHHl LOG 



No* Qttestlonnalre 

I 



Introduction to 
Sinolation 



General Description 

What mterialB were used; effectiveness in 
terms of student understanding and. interest > 
technical quality^ suggestions > etc. 



II, 



Module Preview 



What xoaterials vere used, effectiveness 
in tenas of student motivation^ tecbMcal 
quality, etc. 



Ill- 



Preparation Hiase 



Similar to above questionnaires with the 
addition of questions regarding integration 
or fit with the rest of the module and 
questions pertaining to the role selection 
process. 



IV. 



Participation' Riase 



A questionnaire similar to a daily log 
Therein teachers primarily identified 
student and teacher problems in getting 
tasks done. 



V. 



Susmary Riase 



Questions relating to the suizaiary in terms 
of it being a reasonable culminating 
activity, etc. 



The General ModtUe Evaluation questionnaire solicited teacher 
opinions of the module as a single entity through questions related to the 
overall adeqtiacy of materials, the sequencing of materials, module 
ixqplementation, student petrticipation and learning, and recommendations . 
The first severaJ. pages of the questionnaire dealt with teacher azid 
student background. 
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Teacher Ppgt Module Bmal Revley 

After a module vas coopleted, the teachers nho had perticlpated 
in the pilot test vere ccxnrened to dlscuBS the aodule« Bsr each 
individual section of a nodule teachers vere asked about; the parti«* 
culsr strengths of that section; the veaknesses; dassrocm solutions 
they used to overccne veaknesses; and vhat recosoBendations or suggest- 
ed changes they* had for revising the mod\ae« aQhasis during 
the review vas placed upon probing into their perceptions of the 
Module and looking for ccxuensus aaong the teachers* 
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III* Restats 

C-6 Student Questionnaire - Collated Open-Ended, Responses to Questions 

#30 and #31.* 

30. Name some of the things you liked most about the simulation and some 
of the things you liked least about the simulation. 



Liked Most Liked Least 

Filling out questionnaires Reading the books 

I liked everything 

Being on W 
The Simulation 

I liked most helping and 
understanding better 

The TV shows 
Being counselor and 
making up people 

Planning the progra;n and 
ordering equipment 

Glides 
Video- Tape 

Introduction 



Helping get together 



Simulating 
Directing 



Job application form 
Interviews 
"Tomorrow Show" 




Tests 

I can't explain 

Doing the post test and 
getting confused 

Evaluating the program 

Shortness 

These things I am filling 
out now 

Watching the football picture 
at the end 

Get out of gym 

Too short - wanted to 
perform roles 



30. (Continued) 



Liked Most 



Liked Least 



Writing out forms and 
making out the plans 



Film strips amd 
booklets 



Planning things 



Filmstrips, booklets 



Working without Mr. Young, 
working in groups 



Being on the audio 
visue^. machine, doing 
the talk shov. 



Teacher, counselor, 
principal 



Janitor, secretary 
libraxian 



The talk show sort 



The writing part 



Counselor, audio visual, 
school teacher 



Custodian > personnel, 
researcher 



It shwed me things I 

didn't knov 

(no response = k) 



Nothing 
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31. Write down some of your ideas on how the simulation might be made better. 
More things like the "Toirorrow Show". 

More time to think about your job to understand it. To work with 
more instructional materials. 

Have more visual aids. Put more of the forms in the packets. 

If the class was longer cmd we could try out the jobs we picked 
and had more booklets to fill out. 

If they all had more and more equipment to help people understand better. 

At first if they would tell you what you were really going to be doing. 
I thought that going on a space ship was a little babyish - it could 
have something else, not a space ship. Possibly have a few more people, 
not just 9 - more like 15 or so. Maybe it could be an elective, and not 
during a class, and tell the people what they are going to do. If they 
may tell you if you wamted to know, what was going to happen the next 
day. 

More time to try each job. 

I didn't mind doing this for you and I hope I can do it again. I 
thought it was great! 

There is none. 

It vas perfect. 

Where we could play our roles instead of just picking it. 
More organization. 

More films - better films - longer and go into the program better. 

By showing more video tape reels 

You could show more movies and give more work. 

No homework. Teach a trade in the class. 

Have more to it. More films or filmstrips. 

We were pushed too fast into new unit of the program 

and didn't know what was going on. 

Have better apartments, more rooms, more stores, 

more workers, bigger space scripts. 

Don't have any. 

You could have done more things than filmstrips. 
Better information on the space ship and how you 
will do your job. 
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III. Results* 



E.II - Teacher Post Modu.'.-- - Panel Review Report 
Panel Review 



Title of Module : Planning Educational Programs 
Ie^ : Jefferson County, Colorado and Denver, Colorado 
Panel Leader ; John Radloff 
Panelists ; Bert Fish, Jerry Forkner - Denver 

Alys Soulier, Roger Young - Jeff Co 
Observer Participants: J.W. Altschuld CVTE 

A.F. Terry, Jr. CVTE 

Date(s) Panel Met: December 6, 1973 and December 11, 1973 
Number of Hours; (3+ Hours Total) 



♦Interpretation has not been provided. 
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SUMMARY/OTHER C0MI4ENTS 



Teacher Evaluation Instruments 



1. Teacher Log: (All agreed) 



- was confusing a»id too highly structured. 

- it was difficult to distinguish between parts. 

- if specificity is to be retained there the logs should be unit 
specific. 

- teachers recommended a more open ended approach (diary). 
2. Module Questionnaire 

- good instrument which appeared quite satisfactory from teachers' 
point of view 

Teacher and Pupil Preparation 

1. Teachers should actually perform in a simulation setting. 



Consideration should be given to 
the simulation they are to test. 

Students should have s:*me "hands 
introduction to simulation. 



teachers actually working through 
(all agreed) 

on" experiences as a part of their 
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IV. Reviser's Information Sunmary (RIS) 
A> Description of the Summary 

The Reviser's Information Summary wais developed for the ptirpose of 
assisting revisers to assimilate information collected during the pilot test 
of a module. To acconplish this, information from each source available 
was first revieved cmd then only major thrusts or ideas from the source 
were suimnarized. (These key thrusts or ideas were determined by the judgment 
of the authors of this evaluation report.) The summary was then transferred 
to the appropriate location on the large sheets which constitute the RIS. 
Lastly, each column was studied and trends were dravn and so recorded at the 
bottom of the sheet. In ascertaining trends the authors used their familiarity 
with data^the module ^ and the data collected. 

In generai there will be one Reviser's Information Siamaetry sheet per 
part of the module and one-two sheets covering the overall nature of the module. 
On sheets which pertain to module parts, only some of the data sources provided 
infomation pertinent to that part. Hence, the sheets do have some blanks or 
missing data cells. The reviser should exercise extreme care in interpreting 
the information on the sheets eind should always keep in mind that comments 
on the sheets represent only a siomraary of key points. In addition, it sometimes 
was most difficult to determine a trend in the information obtained. 

B. Use of the RIS 
f ne way the reviser might use the RIS is as follows: 

1. Read the module - become thoroughly familiar with it; 

2. Read the first part of this report (Sections I and II) 
thoroughly. Skim the results compiled in tables 
(Section III, parts A, B, C, D and E.) Read section 
E-2, the teacher panel review report, closely; 



ERIC 



60 



Read and study the Reviser's Information Stammary. 
(Consult original data sources, if necessary.); aai 

Generate a set of revision specifications based upon 
knowledge of the module, the Reviser's Information 
Sunnnary, project developmental criteria and other 
information, if appropriate. 
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C. REVISER'S INJORMATION 
SUMMARY 
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QATA 

SOURCE 



STBENGTHS 



Education Module: Ovari 
WEAKNESSES 



to 



CO 



On a highly reliable achievenent test* the 
experimental group of students gained nearly 
2 1/2 points as their mean score increased 
from 26.1 to 18. 5* (Out of a total of 1*5 
items)* The gain was high in relation to a 
comparable control group* Further specifica- 
tion of this gain indicates that the majority 
of the change was occuring on questions re- 
lated to the educational planning process. 
(For further details see Tables III-A-3 and 
III-A-U J 



Students did not show any appreciable gaic 
related to Job responsibilities in educatj 
ly was influencing their understanding of 
not their understanding of specific occupi 
Tables III-A-3 aud III-A-^*.) Study of att 
dicates that there was no overall change i 
preferences and number of reasons given bj 
Further probing of the type of reason give 
similar patterns of response for both expe 
groups. As noted in Table III-B-5* seven 
could account for these results* 
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M 
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There were strong positive resporises to all 
questions about the module overall, as in* 
dicated by the following percentages; 

Learned about Jobs ^6^ 

Wasn't boring 7^ 
Like to do another simulation 65% 

Answered Job questions 65% 

Not too long 70^ 

Not too complex 7^ 

Instructions led OK 36^ 

Not too many tests, forms 61% 

Fits together well 6li 

The activities which the students reported 
liking the most were the planning, talk shew, 
and writing out forsis. 



Students reported that the filmstrips, boc 
the least liked elements. 



CO 



In general the reading level was OK for 
the students and the module was considered 
appropriate for 8th and 9th grade students. 
Students were receptive to simulation 
ind to the module content and found parts 
of the module exciting* 
The quality of the module was rated good 
to very good. 

One teacher noted an improvement in 
teacher-student relationships. 



1. Two of the teachers felt that transiti 
the module should be ioproved. 

2. Two teachers mentioned that interest 
time went on. 

3. Hard for kids to follow continual shif 



63 



•rail Considerations 



RECOMMENDATIONS FOR BEVISION 
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alns on knowledge questions 
at ion. The module apparent- 
of planning in education but 
apations in education, (see 
attitudinal test data in- 

in strength of the studert 
by students for a preference 
Lven for a preference reveals 
cperimental and control 
^ral different explanations 



Evaluator's tentative suggestion for revision* 

1. Module emphasis should be more firmly defined, i.e., is it an 
educational planning simulation or a simulation dealing with *'try 
ing on'' educational roles (teacher, administrator, etc.) If the 
decision is made to emphasize educational roles, then more ex- 
panded detailed descriptions of those roles would be necessary. 
(As i^i noted in this summary, the descriptions cannot be sterile 
in nature and still be affective.) 



)ooklet8y and tests were 



The students suggested more films of a better ouality. videotapes, 
better instructions, and allowance of more time to play roles. 



tion from phase to phase of 1. 
and attention decreased as 2. 
Ifting of roles. 3. 



Provide more informtion and structure in planning program and 
evaluation of program. 

Teachers Should be prepared by going through the module Just as 
the students will. 

Make-up of the group should be recognized as i«portant; a homo 
geneous group of low-achievers or those with little motivation 
would have difficulty. Important for the group to work together. 
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DATA 
SOURCE 



STRENGTHS 



Education Module: Overall ( 
WEAKNBSSES 



Q 



CO 



It is clear ftom all of the above sources 
that the nodule was successful* Students 
gained much cognitive information as verified 
by their perceptions and test scores* Ad- 
ditionally, they felt the module fit together, 
wasn't too complex, etc., i.e., the module 
"was making it" from the student viewpoint. 
The student perception of the module is con<- 
firmed by teacher comments in the LOG/General 
Module Evaluation* '^he teacher comments 
strongly parallel stut^ent responses. However, 
there were weaknesses across the module as 
well in specific parts which have been noted 
in the RIS pertaining to those parts. For 
example, teachers felt that there were seri- 
ous problems wich Task ^ especially as it 
related ^;0 the reat of the module. 



A strong pattern in regard to roles and op 
role emerges. Students simply did not inc 
of Job responsibilities and they commented 
to really experience a role. Teachers als 
of transition betwmen module parts and the 
had in continually shifting roles. This 
in a decrease in interest as students prog 
There were major difficulties toward the e 
under Task ^, Task ^, and the Summary. 

Also problems with booklets, media, etc. a 
where they occurred in the module. 
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U Considerations 



RECOMMENDATIONS FOB REVISION^ 
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1. Teachers should work through the simulation themselves # so that 
they have a better understanding of the module before they start 
teaching it. 

2. Students should be introduced to simulation with some "hands on" 
experience. 



apportunity to enter into a 
iacrease their understanding 
tid about the lack of time 
also commented on the lack 
tte difficulties students 
i may have manifested itself 
rtgressed through the module, 
end of the module as noted 



As noted above and as noted in the sheets defining specific module 
parts, the reviser will have to make decisions concerning the roles 
included in the module and the amount of time to play those roles 
effectively. This will probably necessitate a restructuring of part 
of the module. Students and teachers were in agreement with regard t 
the need for improved instructions, more Information and a clearer 
idea of structure (perhaps intent) and form of the module. It was 
suf^gested that the introduction to simulation be a somewhat more acti 
"hands on" experience. 



. are noted in the section 



With regtrd to facilitating the module, teachers felt that they shoul 
work through the entire simulation prior to leading students. Teache 
expressed concern whether different types of students could success- 
fully use the module. 
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From an incremental test done in the Fall of 
1973 the following results verc obtained* 
87^ (N=15) or more of students using the 
materials felt that they understood the 
materials and that the vocabulary was 
easy to understcmd. 



1. Both the slidea «Dd the booklet wre 
rmtad Ttry good at to technicAl qtttll^, 

2. Tbe stttlettts vere able to vnderstand the 
concepts preieoted laost of the tine ud 
Here stiflolated l^y the preaentatlon 
scnewhat* 



Education: Introduction to 



When students were questioned with regard 
enjoyment with the introduction, the qual 
etc. the picture beceune somewhat more mix* 
-Cnly 53^ of the students were firm in t 
statement of enjoying the booklet or th 

-About 1/3 of the students were strongly 
liking the illustrations. 



1. 
2. 



OxK teacher ccanented that tbe student 
eepte to the task* 

It ims too sliple for the 8th grade* 




Had no effect at this time; not deslgnc 
did not repeat key ideas emphatically e 
Role Identification process not entlrel 



Both students and teachers seem to agree that 
the concept as presented in the slides and 
the booklet did come across to those using 
the materials. Biere was some difference in 
perception regarding the quality of the 
illustrations. (See Student Questionnaires- 
Weaknesses . ) : ^ ^ 
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First, it did not seem to be as motivating 
should be. Students had mixed feelings abc 
Teachers also identified the same type of j 
the materials did not emphasize or repeat 
emphatically enough. Pelated to this is tl 
the introduction with the module that folic 
students did have mixed feelings about the 



;o SiKulation 



"■'r:M a: 1^:;:' .-^n KEvi:;ion 
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to their overall 
ility of the materials, 
jced in nature, 
their 
he slides 

y positive m terms of 



'Slightly over one-half of the students recommended that the 
slides and bookie i. be used together, with the slides coming first. 



its did not relate tbe ecor 



led foi first €u:tiYity; 
enou^. 
feiy clear. 



11. 
'2. 

!3. 



Repeat ideas. 

Needs to be active rather then passive; use prograjBued instruction 
types of ideas. 

Include in preparation or another handbook; transfer concept of 
siimilation into role preparation. 

Have a slide towi\rd the end that asks what siawlation is and have tl 

student write tliat definition to see if it needs more reinforcement, 

Use color in book . 



; as a first activity 
out the presentation, 
problem. Secondly, 
key Ideas 
;he problem of integi 
X"^ O '-^stly, the 
^ERjQrations. 



ting 



In the recommendations above there is a strong en?)hasis on 
repeating the key ideas more than once. The students and teachers 
both seem to be commenting about this. (Especially see teacher 
recommendations.) Also the reviser should pay careful attention 
to the integration of the introduction with the rest of th** module. 
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Education: Module Preview 



DATA 
SOURCE 



:'.trength:' 
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Sixty-five pereent of the students felt that 
the preriew helped to prepare for the 
sinOation* 



Tbn 8lide->tape was rated very hi^ 
with conoents that the imsie ims good 
and the educational qoali^ outstanding, 

Cne teacher conmiented that the pupils 
liked this activity. 



1 MUsic good 

2. Interest picks up here (unanimous) 



1. Poor quality of tape (not the voIceT^wi 



a 



CO 
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From both the viewpoint of the teachers and *~ 
students, the preview slide/tape received 
high ratings with especially positive com- 
ments on the music and on the i nterest -genera 
ti.ng qualities. Teacher coar^ents on these 
points were consistent between the logs and 
panels « 
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The only weakness noted reflects on the po< 
tapes sent out. This could be prevented hj 
fore delivery to the field. 
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.-OK kEVLSIOII 



1. TWO of th* three teachers favored dropping the 
slides of "today" people in faror of a real 
science fiction approcMSh. 



was noted c»ie ' teacher . 



1. 

2. 



Let preview provide acre of a lead into sinulation tasks. 
One teacher suggested using all futuristic slides. 



poor qualify of one of the 
z-y checking all tape? ce- 



ERIC 



The following zninor suggestions for improvement were 
offered by the teachers: delete slides of "today" 
people and substitute for these futuristic or science- 
fiction types of slides; also add more concrete leads 
into what the siailation tasks will be. The teachers 
were Consistent in their comments. 
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DATA 

SOURCE 
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1« Urn slide-tape vee rated hie^ in technical 
qmlitar^ but not as blc^ in educational 
qnality as the preriew* 

2* The booklet vas rated high in technical 
quality also. 



!• Visual of the space ship showing decis 
turned kids on. 
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Education: Preparation 



WKAraiEGSES 



1. ne slide-tape needed more detailed ex 
>fork« 

2. The vocolmlary level was too simple^ 

3. Trouble understctndtng roles necessitati 
direction. 

U. The booklet was rated meditm in overal 
because of confused directions and ina< 



1. Cony TocahulGLry ("hi, John") 

2. Break in flow between preparation and 



In the judgement of teachers, the slide tape 
had high technical quality but there vrere some 
weaknesses in educational quality -v/hich led 
them to tiudge it inferior to that of the pre- 
view (see next column). This should be 
attended to in revision. 

The visual of the space ship tos regarded by 
the teachers to be a strong point of the pre- 
paration and no changes were suggested* 



1. Teachers noted that on the slide tape 
was needed and that the art work could 
other comments made by the teachers, i 
probably relates to what the students 
simulation and, specifically, their ne 

2. Rereading the handbook confirms the te 
a DreaJc in flow between the preparatio 
task. Specifically, there are no dire 
indication of the role(s) he will play 

3* Teachers commented about simplistic an 
most likely shoxzld be altered in revis 
whether this coranfint pertains to only 
slide/tape and handbook. 
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\i'.:rr:rn^ATio:jr> i-ok revision 



60 



escplAiiatioQ ud better art 



ited teacher 

111 educational quality 
ladequate explanations. 



id first task. 



1. Use the space context for teacher survey and later interviewing^ 

2. liiprove the wording on the slide tape. 



more detailed explanation 
Id be iiaproved. Judging from 
the explanation requested 
; are to do throughout the 
aext step in the siiaulation. 
jeacher comment that there is! 
Ion handbook and the first i 
ections for the student or j 

tnd corny vocabulary which 
sion. It is not clear 
r the slide /tape, or to both * 



1. Since there is only one source of information 
on this point, no trends can be established. 
However, the teacher comments suggest that we 
could capitalize more on the space context for 
later surveying and interviewing and could im- 
prove the wording on the slide /tape. 
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Bducation: Task 1 Identifying People and 
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Sixty-five percent of the students liked the 
role selection. 



2. 



RecoHnexided tijne was appropriate. 
Had a good time with it. 



Has potential to get kids into exploration 
prooess* 
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"Both sttidents and teachers seemed to 
react positively to this task. (It 
should be noted that since there are tvo 
role selection points in the module, it 
is unclear to which of these the 
students were referring. ) 

Apparently the survey technique was fun 
for the students and has the potential 
to get them into the exploration process 



lo Teacher had to work closely with stude 
2« Students dldn*t seem to grasp purpose • 



!• Wotrding and direct:* t^ns confusing for h 
learner (pages 7 ic 8); word '^consultan 

2. Curriculutt should be selected before t 
that data is not used to pleui prognua; 
handbook are not clearly separated* 

3o Teachers felt need to Interrene to bel 
and to point out things beyond the stu 
(i.e., need for adult education) 



Both in panels and on logs teachers pointec 
fusion. This is evidenced hy the fact thai 
they had td intervene frequently and that 1 
was not clear* Confusion seemed to center 
wording and directions (exa!!i"»ne pp. 7 & 8); 
data collected is to "be used in other 
the actual tasks involved in canying out 
teacher recommendations ) . 

A second problem relates bo whether or not 
point out things beyond the student's pers 
example ) . 
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ii Coorses 



K^^.v^L^E^^■AT^o:^s for revision 
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dents to avoid confusion. | 1, M&ke it acre purpoge dircctedo 



— u 



both facilitator and 
int'* lalsuiider stood Iry soae 
the staff; It is unclear 
t; the tvo tasks in this 

ap stodents f^quently 

;udents present perspective 



^1, 

12, 

j 

i3o 



Use role or task cards for direction* (esp^ for 
researcher^ consultant )» 

Divide task one into such things as setting up 
ftoilies, o«ducting the surrey, tabulating re- 
sults, and have more instjructions on each^ 
Give soM structure for the data (such as a 
form listing classes for Ysrioos grade levels). 



5d out that there i-ras con- 
it the teachers felt that 
the purpose of the task 
on several issues: the 
J* the idea as to hovr the 
siinulation activities, and 
the svjr/ey (see specific 



: teachers shcmlrl inten/enej j 
pective. (See above 



The teachers* comments regetrding weaknesses in 
Task 1 consistently lead to the recommendations 
that have been made above in this column. The re- 
viser should continue to use the activity of sur- 
veying a hypothetical community, but build in a 
clear subdivision of activities and their sequence 
with clear directions for carrying them out. 
Attention should be paid to how the data collected 
in the survey would or could be used in later tasks 
end to pro\''ide better forrrjs for the students to use 
in analyzing tha data. 
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EAttc&tlon: Task 2 Deciding on St 



DATA 
SOURCE 
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Si3cty-five percent of the students liked 
the role selection. 



The tine «uid level were appropriate fbr 
the stttdeats and the task integration 
vas ok. 



Video tape on laterTievlng is excellent* 



lo Students didn't know vhere or how to 

2o Cne teacher felt that charts and exi 
helpful. He drew an <rgani7ati«n cl 
duties of various non-teachlmr ^tafl 
students would know more about actua 
Another teacher resorted to a 1-2 ni 



1 . SUdertape presentation is boring^ sta 
. Hkxd for kids to follow costlmial shif 

3. Some roles may not fit the spaceship ( 
others (learning theorist) are diffici: 
to understand. 



Seemn^ly hoth the teachers and students re- \ 
acted favorably to the task as a whole . student^ 
were positive about the role selection process.! 
Teachers were especially enthusiastic about thej 
video tape on inte^nrLe^-rLng, terming 5.t excellenjt. 
They reported that the recommended tme \ras 
appropriate for the task and that the level of 
the materials was suitable for the students. 
Mowever, the teachers did note areas for possibfLe 
improvement as detailed under "vreaknesses 
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Vrnile the students seemed positive t 
process, teachers expres.sed several 

1. The slide tape presentation dep: 
\TB,s judged to be sterile and hoi 
worthy when ca^apared to the exce 
videotape or interviewing. 

2. Teachers commented on the diffic 
tinually shift roles in the simi 
sLniulation shows that in Task 2 
into their third role. Prior tc 
been in the roles of community n 
consultants . In Task 2 they bee 
committee for the space ship. 

3. Students did not Vjim how to sts 
one teacher noted that more chai 
(Perhaps better directions for j 
session 6u:e required). | 



1 



staff Needs 



o2 



to start. 

examples v;ould be 
cha»*t and t utlinea 
iff mmlers so that the 

tual school operation, 
niombering system. 



1. Need znore structure perhaps an example o 

2, Pupils need more th^n » «?ter5ie job c'escription for all positions. 
3- ''iive more information about actual operation of schools and 

duties of various staff mambers (teaching and non-^.eaching) . 



itarile. 

lifting of roles. 

p (librarian, custodian) and 
Lcult for teachers and kids 



e about br.e role select ior. 
al .'u&Jor concerns • 
Lctin^ educational roles 
rin^. This coirunent is note- 
xcellent rating given tc 



students /^to con- 
Reviev of the 



xculty or 
isiulation. 
2 the stmients are entering 
to thic point they nave 
Y raeinbern and researchers or 
jeconie *»ie7xers of a pianninjs; 



Teachers recommendations are consistent vith 
teacher comments regarding the need to give 
more information about educational roles as 
veil as improving the interest value of the 
materials presented . For example teachers 
riaggested more emphasis on the actual operation 
of schools and dutie.s of various teaching and 
non-teaching staf^ mambers roles* It \m.s also 
sur^ested that we reconsider some of the 20 
roles included in the occupational descriptions. 



tart 'i'ask 2. In addition 
larts and examples are needed* 
conducting the planning 
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DATA 
SOURCE 



CO 



< 



►3 
s 



g 




1. The tiioe and level were appropriate for 
the students. 

2, Integration with the other tasks was OK. 
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Education: Hiring tbK^ Staff 

Task 3 



lo students couldn't begin on their own^ 
ceedo 

2p Students had no knowledge of Job requi: 

3. There was difficulty getting students 



2. 
3. 



Only competition for a role prorrokeA i 
should be needed for all jobs. 
Not eosmt^ directions for filling out 

Not enough cues to respond to role-pla; 



The teachers* corninents on the logs indicated i 
that the time for the task, the level of the 
the materials, and the integration Tri.th other 
tasks were all appropriate. In general as 
noted in the next two columns the teachers 
seeaed to focus more on changes to he made. 



1. 



Students did not have enough knovrledge 
directions for filling out application 
Students had difficulty beginning or u 
task^ especially with I'egard to gettini 
haps more directions are needed.) 
Interviews for all jobs seemed desirab 
teachers . 
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6? 



if cozifused about how to pro- 

uireznents* 

s into ijoles. 



Send pupils to interview school personnelo 



•A interriew; mldL Interyiewfl 
ut applications* 



1* 
2. 



Have IdLds fill out applications independently* 
Send kids to places in the educational setting 
(couaselory principal^ etc*) to find oat 
about idiat they are applying for* 



ige oi ,jor require.nents or 
ior.s for the job . 

mitiating activity in 



Vciiz 



bins into their roles (?er-» 
rable in the judgement of the 
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Teachers commented consistently on both panels 
and logs that a good way for students to obtain 
information regarding roles in the educational 
setting is by simply intearvievring school person- 
nel. This activity could easily be incorporated 
into the sinailation. 

l^ovide competitive setting for students to get 
into roles so that interviewing becomes a necess- 
ity. One solxition here might be to have students 
in d^pe nd e r>t ly apply for the same job. 
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DATA 
SOURCE 



CO 



CO 



J3 



Q 



CO 

O 



The tliae aivl Icrel were appropriate for the 
8tudemt8« 



Planning the facility is o.k. 



Education: Task k Planning FMilitlt 



1. Students didn't see wxch pxirpose; no ejc 
2o Students "hired" but never got to *'wor]ii 
going downhill. 



No budgetary constrajjits. 
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As noted above, time and level seerrjed appro- 
priate to the teachers, "but some weaknesses 
were noted. These weaknesses becorae niore 
apparent when tasks k and 5 are jointly re- 
viewed. The simulation seems to be falling 
down at this point. 
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Lack of student perception of purpose oi 
suited in insufficient motivation, 
students x^ere hired in educational rolei 
ooportunity to experience or engage in 3 
Budgetary constraints were not given to 
planning the facility. 



r 



ities and Supplies 



M*^rr::;i ATio:;r> for revision 



6a 



) laotiYation, 

rork;** things really began 



Budget limits need to be built in* 



Build in budgetary restriction in planning the 
facility (use large budget and stress loading up 
before space trip). 



of the task, perhaps re- 

les but never really had the 

role activities. 
to studentH for use in 

ERLC 



The only recommendation / i-j I .,uild in 
badjetary restrictions » (see above) 
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'TrvKIJGTH: 



Education: Task ? EvaLuatlBg the Scl 



I 



The leTcl was appropriate for the atudenta 



Potential to tie together the researcher/ 
consultant roles tram the first task vith 
the evaluation here* 



I, 
3o 



lo One group needed more tine. 
2. Students showed lack of interest in era 

because there had been no challenge • 
3* Confusion on directions « 



AJUL teadtert agreed that unclear direct 
in shifting to eralnation rolco 
Puxi>o8e8 for the task inadequate, no n 
Students not led to the tram of mind i 
paraaeters* 
Urn evaluation forms: 
-were not consistent vith handbook dire 
-were ndssing criteria for use of ^ poj 
-used criteria about which no previous 



l/hile teachers comiaented in the iocs that thel 1. 
level of the task vrats appropriate for the | 
students, the thrust of conments from the logj: 
and panels indicates that ther<^ is potential 
for meaningful evaluation v/hich was largely 
unrealized due to the difficulties expressed 
in the next column. 
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There seems to he a trend in teacher co 
no c learc ut idea tliat they were ever to 
planned. Since the students did not ha 
of the importance of evaluation, they 
accept the evaluation parameter? and pro 
teacher observed that the lack of inter 
have resulted from the fact that studen 
a ''neat" thing to do rather than as a c 
-"directions for accomplishing task 5 wer 
ted to the proble.'ns the students had in 
role . 

The "hov; to*' of eTOluatiof>(the criteria 
specified in Task 5. l^y special atten 
'\y^y Y''^* ' consistent with handbook 
'^1'-*:". .'.wT. v£i: 2iven in the handbook fo 
... . fivjind on t'le ''?rm. 



ehool Progrma 
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I 

t 



♦ Form sliduld ^ 'revifled. 



i eraluating themselves perhaps 



jrections caused difficulty | 

10 reasons for stxident ratings 
nd to accept the eraluation 



One teacher felt that the last page of the" 
bookliet should be part of the evaluation 
instrument o 



directions 
point scale 
.oas V6iming bad been given. 



tr co:r.-.ient;: that ctudents had 
r to evaluate the school tney i 
ft have a full understandinc 
y found it difficult to . 
pro cess. ; 'ore over , one \ 
nterest in evaluation rrji^y I 
udents perceived the module a^: 
a ch&llen^re. 

were not clear anc' contrihu- 
a in shiftin:: to the evaLjation 

ria) is not well defined or 
ttention to the fact that the ' 
ook i root : ons , i . , * o 



^e.qo^erdaUons: 1. Teachers noted that there is potential for tyin 
o-ether tne research roles (Task 1) and the e/aluation (Task 5) ? 
ra^p-. provide increased integration, give the students early warning 

S'lien? ISrl^ttrr^''^ ' '"^^^ '^'^ "^^^^ ^ 

^' ^it^^rl'f ^^T^^^"^ P^ee of the booklet should be 

part, of the evaluation instrunjent which my be indicative of a nei 
to improve the directions. xi,auive oi a ne< 
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DATA 
SOURCE 



.■.TPK:;r,TH:' 



Education; Stnoaxy 



CO 
Eh 



CO 



CO 

§1 

Eh m 
CO Eh 
CO 



CO 



J3 

(xl 



Eh 



CO 



; J: :7'SL: percent of the students felt that 
the stuaoary helped to "pull things together." 



One teacher \ras especiall:'' enthusiastic about 
the Summry and conner^ted that it was very 
effective, well integra-'^ed with the other 
tasks, and that the kids really looked fortv^d 
to this . The other two teachers were less 
positive, and their conL;ients are noted under 
weaknesses . 



Students hlj^ly Motivated by "TcBaorrow 



One *^3acher felt that the Ctumnary was not 
rest of zhe simulation ar.d that it hore nc 
roles the students }iad played. 3iscussior 
"social** aspects of life in space rather 1 
teacher noted had been (Hscusced earlier i 



A majority o'f the students 'felt "tl:aVThT 
sumoary helped to pull thi>:gs together, 
and one teacher connaented in the lo^ that 
it seeiced to be a very effective culninat 
^o the simulation experience and that the 
students looked forvmrd to it. 
There ^ras disa^^reenient a.TionG teachers as 
to how well the Zurx^irj \vras intejrated 
v/ith the rest of the sL-nulation. .'^ne 
teacner saw it as fx st.ren^ 
a uealcnesG . 

Teachers indies t^-^d in panels t'lat tne, 
students were h^-r.ly -lotiva^ed by tl.e' 
'Toniorrov; 3how"; lovevor, ^hey poir.toc 
significant weaknesses that deniand re- 
vision attention. 



Ic 

2. 
3o 
K 

5o 



I 



thy another 



Little eontinuity with rest of siaolatj 
Net enoo^ tide to do show vell« 
Directions for host not e3q[>llcit enooj^ 
!Sie guests had pex*f omed the same task;! 
say that vas nafv^ had to fake their s^tt 
Didn^t go l>^yond Sections I and II Imm 
lead them further* 



n possible lack of continuity* with the 
bothered one o:^ thu teachers. 
T::ere aTIs roi enoujh ci:ne to do the tal 
teacher panels, and students ten.ded (pe 
lack of tizis} to discuss "social'* asoec 
than role -related inforioation. 
Teachers felt that the directions for t; 
explicit enoujh and 'cl-iat the host lid m 
f'-iiTcusKion of . octio!i III the rue*;- 
Teacncrs co..iLjCi.,tc:d in pcnelc thcx^ 
h^.' r^-:r"or'.;efl tho .'^.-^^.e t^'.fr"-,'?, .so in''^V'i< 
culty findinc i:o:aethin3 to say t';at had 
t-ioned . 



oi: 



1 
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66 



ot V7eli integrated vitli the 
r.o relation to the education 
ion seer ci to center on tne I 
r t:.dn on roles (vrliich anotheij) 
r in the i:.tervievs\ , 



ition, 

Iks 9 so some had little to 
lehool rolea. 

Kiawse host questions didn't 



Mi^t use the who, what, when, where, how 
approach for the hoato 



ali: srio:;^ accord to ;/:e 
rerhaps partly recause oi 
ects of ijoace life rat /ir 



1. 



a* Ox a car-: . 
id not 1" ready been --e 



^ne lueotions of Lntegratioa vath the sLmulatiorf 
ant tirae allotted should be examned and re- 
solved . 

The host* 3 directions should be aoiplifled and 
T^de .-nore explicit, it vould be desirable to in- 
yvTo 'hat the host ^juides the Jtudents through 
e-n ovf-'r\^iew of tne ci-ulation exDeriences (parti- 
cu.l?r3:- \:±lh rerer^^nce to the roles played) and 
vjiat ':ue':t:ons ai-e asked that provide each stu- 
cient wiofi 1^.^ Oy^wtUiait^- to ^ive a ajeanin^fui 
resyonoe. A ':i;:;;.jestion from the te-^cher pan*:*}? 
\:hi(:- r.ioulr! be considered is to use the v;ho, 
•'hat, ^t.en, v/iierc and hov: approach for the host. 
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APPENDICES 
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APPENDIX A : 
Kir.WLED^ TEST - "WHAT DO YOV KNCW 



Planning 

Educational 

Programs 

•What Do You IQiow?* 



The project presented/reported herein was performed pursuant to a grant 
from the National Institute of Education, Department of Health, Education, 
and Welfare. However, the opinions expressed herein do not necessarily 
reflect the position or policy of the National Institute of Education, 
and no official endorsement by the National Institute of Education should 
be inferred. 

Copyright 1973 by the Ohio State University, The Center for Vocational and 
Technical Education. 

Copyright for these materials is claimed only during the period of develop- 
ment, test, and evaluation, unless authorization is greuiced by the National 
Institute of Education to claim copyright also on the final materials. For 
information on the status of the copyright claim, contact either the copyright 
proprietor or the rJatio.ial Institute of Education. 
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PLANNING EDUCATIONAL PROGRAMS 
"WHAT DO YOU KNOW?" 



The purpose of this test is to help us find out what you and other students 
like you know about the planning of educational projects. This test does 
not in any way affect your grade. 

DIRECTIONS: To complete the test first fill in the information requested 
at the top of the next page. For most questions on the test 
there are several short phrases or statements^ TIsted. Pick 
the one that best describes your answer and circle the letter 
in front of it. For several questions special directions will 
be included with the questions « Please follow those directions. 

If you don!t know the answer to a question, GUESS. When you 
have completed the test return it to your teacher. 

Thanks for your help. 



✓ 



vuu may turn the page and start as soon as you have completed reading the 
di rcct ions . 
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PLANNING EDUCATIONAL PROGRAMS 
"WHAT DO YOU KNOW?" 



FILL IN THE FOLLOWING INFORMATION. 



Name 



Date 



School 



City 



Age 



Grade (circle one) 8th 9th other (please specify) 
Sex (circle one) Male Female 
Subject taught in this class 



START THE TEST 

1. What of the following should educational programs be designed to serve roost: 

a. The U.S. Employment Service 

b. The community in which the school is located 

c. The faculty of the school 

d. The Board of Education 

2. What information included in a community survey would be most beneficial in 
the planning of school curriculum? 

a. The job plans of high school seniors 

b' ThG ages of the residents and their long range educational goals 

c. The number of college graduates in the community 

d. The hobbies and recreational interests of the majority of the community 

3. Thu English Department in a large high sctool feels that they are not meet- 
ing the individual learning needs of students. Which person would the 
principal most likely contact to v;ork with the Department? 

a. Counselor 

b. Curriculum Supervisor 

t;. Director of Audio-Visual Services ^8v\\ 



d. 



Registrar 
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4. What can educators do in order to decide on an educational program for their 
comxaunity? 

a. Look at the programs of other communities to see how satisfied the resi- 
dents are 

b. Collect the opinions of educational experts and the Board of Education 

c. Survey the community, talking to a large number of residents 

d. All of the above 

5. When a person is applying for a job in a school system, what information is 
considered before the applicant is hired? 



a. Information provided on forms and transcripts 

b. Ir.formation provided by character references 

c. Information provided in an applicant interview 

d. All of the above 

e. a and c only 

6. ;Vhich of the following tasks is most likely to be done by persons in charge 
of planning educational programs? 



a. Preparing a curriculum outline 

b. Preparing lesson plans 

c. Preparing athletic schedules 

d. Preparing tests 

7. A large number of Spanish-speaking Americans in the coirmunity would like 
their children to learn to read Spanish in the elementary grades. They 
have elected some representatives to meet with the school superintendent. If 
you were the superintendent which would you ask to attend this meeting? 
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a. Curriculum supervisor 

b. Director of audio-visual services 

c. Counselor 

d. Learning theorist 

The Johnsville Neighborhood Council is setting up a Summer Learning Center 
f r Junior High School students. The director of the neighborhood couajbil is 
very interested in this project and wants the Learning Center to be a real as- 
set to the neighborhood. Which of the following techniques do you think he 
should use i.. deciding what will be taught at the center? (Check as many as 
you think would be helpful.) 

Arrange to get ail discarded textbooks from the Junior High. 

~~' Survey the junior high students in the neighborhood to find out what 
they want to learn. 

Plan to build as many facilities as possible and then what can be 

taught in them. 

rleet with the r^chool Njard and arranae to have some courses switched 
to the neighborhood center* 

Meet with parents of Junior High students to find out what they would 
like their children to learn. 

Kind out what kinds of courses are offered in other similar learning 
centers. 

Find out what courses used to be taught in Junior High and plan to give 
those that are no longer taught. 
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9. After an educational program is designed and is being used in the schools, 
educational planners continue to work on it. Which of the following 
should concern them most at the end of the first year? 

a. Identifying people and courses 

b. Hiring the staff 

c. Planning facilities 

d. Evaluating the program 

10. When evaluating a plan for an educational program the most important con^ 
cern should be: 

a. Whether the plan serves the needs of the community 

b. Whether the plan provides for new textbooks every 5 years 

c. Whether the plan includes the teaching of world geography at grade 8 
or grade 9 

d. Whether the plan calls for the involvement of a parent-teacher organi- 
zation 

11. Several teachers in the Rosebud School would like to change to a different 

system of teaching first graders to read. The principal and curriculum .^^^-^ 
supervisor are not sure that the new reading program is very good. Which 
of the following individuals might be able to help them decide? 

a. Librarian 

b. Nursery school teacher 

c. Learning theorist 

d. .Counselor 

12. Wiiich of the following data would be least useful to persons in charge of 
planning educational programs? 

a. The ages of community residents 

b. The ambitions of community residents 

c. The addresses of community residents 

The occuoationB of con»nunity residents 

13. Which of the following steps comes first in planning an educational program? 

a. Drawing the floor plans of the school building necessary for the program 

b. Determining the educational needs of the community served by the program 

c. Detennininq the number of people to be served in the school cafeteria 

d. Getting the books and supplies needed for the prograir 

7'^ 



» 

\ 
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14. East Apple, a town of about 4500 people, plans to open an elementary school/ 
List the order in which you would organize the planning activities by using 
the numbers 1 through 6 only one time each. 



Opening day of school 

Survey community to determine needs 

Order supplies and equipment 

Choose curriculum 

Deciding what staff is needed 

Interviewing and hiring teachers and staff 

. Which of the following factors is important in choosing personnel staff 
a school? 



a. The amount of money available 

b. The needs of the community served by the school 

c. The xiature of the educational program to be offered 

d. All of the above 



In each of the following statements, place a T in the blank beside the statement 
if you think it is a true statement or an F beside it if you think it is a false 
statement. 



Planning educational programs is done primarily by tbi individual 

classroom teacher. 

It is usually important for educational planners to design the 

building before they do anything else. 

Educational planners are more concerned with record keeping than 

with other aspects of school. 

Education a plann-rs can dictate whatever kind of program they want. 

Educational pl-iiners are responsible'for developing a broad outline 

of what will be taught in a school.' 

ySducational planners must take into account community desires. 

22.. John's family is planning a short trip early this spring and John must find 
out when spring vacation will be. His teacher isn't sure of the exact dates. 
Which one of the followincj people i^.hould John ask for this information? 

a., School nurse 

Datri systems analyst. / 
c. :Juf>oriiiLciidont 
d- School secret:ary 



21. 



o 

ERIC 



23. The local Education Association is compiling an up-to-date list of school 
personnel eind their responsibilities for the Board of Education. Some of 
the job titles and descriptions are given below. Match the job title with 
its job description by placing the letter of the description by the job 
title, it most closely describes. (Use each description only one time each) 



Description s 

Is in charge of obtaining and 

distributing tapes, films 

and other media for the schools 

On/anizes and carries on leisure 
activities such as crafts, games, 
hobbies 

Has responsibility for admirxi- _ 
stering entire school system 

Teaches one or more subjects in 
grades seven through twelve 

Issues transcripts and/or school 
records of students* courses and 
credits to future employers 

Organizes and leads activities for 
pre-kindergarten children 

Plans for the preparation eind ser- 
ving of meals to students during 
the school day 

Helps students to solve individual 
problems often by interviewing them 
and their parents 

Takes care of clerical responsibili- 
ties for the school 



Works with teachers p^nd administrators 
to develop courses of study 



Job Titles 
Counselor 

Curriculum Supervisor 

Director of Audio-Visual Services 

Director of Food Services 

Nursery School teacher 

Recreation Leader 

Registrar 

School Secretary 

Secondary School teacher 

Superintendent of Schools 
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24. ^ What part of planning is often done by researchers and educational con- 

sultants in the planning of a school program? 

a* Developing the school curriculum 

b. Surveying the community as to their background and educational desires 

c. Recommending the kind of staff which is needed for the program 

d. All of the above* 

25. Which of the following groups is responsible for operating a school? 

a. Parent-Teacher Association 

b. Board of Education 

c. National Education Association 

d. All of the edx>ve 



APPENDK B; 
ATTITUDE SCALE "WHAT DO TOU LIKE?" 



Planning 

Educational 

Programs 



•What Do You like?' 



r;7 
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PUNNING HpUCATIONAI, PROqPAMS 

WHAT DO you piKt? 



T»n , 15; WOT A THST> The purpose of thQ5« questions is to 
find out the types of activities ancj jobs in the planninci 
of educational programs you might enjoy doinq. vfe wqul4 
also like to learn what reasons you havq for iikiug thes<i 
activities and jobs. 

Thorc are only nine (9) questions to anf|wer» Directlpns 
for ani'.worinq are found at the top of eac)i P9<?e, 

After "ou have completed the questions, please jr^tuvn ^Yiis 
booklet to your teacher* Thinks for your help* 



5 l*^asc beqin the questione soon «^ ypu have fini^heJ^ reading 
the above paragraphs. 
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1. 



What kinds of experiences or activities do you think people should 
have before they select a job in th^ world of work? Briefly describe 
or list your ideas below, 



8. Have you ever thought about how you would go about selecting a job? 

What are the most important things that you feel people should consider 
before they select or decide upon a job in the world of work? Briefly 
describe or list your ideas below. 



9t Pretend that you have interviewed for several different jobs in the 
last few days. Yesterday two employers chilled you ani each offered 
you a job in their organization. Both employers want yot^ to decide 
within two days whether or not yoy are going to accept their offer. 
Briefly describe below how you would arrive at yovpr decision. 



Please return this bookie^ to your teacher^ Thank yoi^. 
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APPENDIX C; 

STUDENT QUESTIONNAIRE - "WHAT DO TOU THINK?" 
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Planning 

Educational 

Programs 

'What Do You Know?' 
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The project presented/reported herein was performed pursuant to a grant 
from the National Institute of Education, Department of Health, Education, 
and Welfare. However, the opinions expressed herein do not necessarily 
reflect the position or policy of the National Institute of Education, 
and no official endorsement by the National Institute of Education should 
be inferred. 

Copyright 1973 by The Ohio State University, The Center for Vocational 
and Technical Education. 

Copyright for these materials is claimed onJy during the period of development, 
tr-st, and evaluation, unless authorization is granted by the National Insti-- 
tute of Education to claim copyright also on the final materials. For infor- 
mation on the status of the copyright claim, contact either the copyright 
proprietor or the National Institute of Education. 



104 



PLANNING CONSTRUCTION PROJECTS 



"WHAT DO YOtJ THINK?" 



Now that you have completed this simulation, the people who developed 
it: would like to find out what you thinJc about your experience. Your 
ideas will help to make the simulation better. Remember, THIS IS NOT 
A TEST and your answers will not be graded. So feel fretj to check and 
to say what you think about this simulation* 

DIRECTIONS: To complete the questionnaire, first fill in the information 
requested at the top of the next page. Then there is a list 
of statements which describe a feeling or an idea about the 
simulation just coirpleted. Answer each statement by circling 
the symbol which best matches your actual feeling: 

(+) means the statement agrees with your feeling 

(?) means your 're not sure how you feel about the 
thing mentioned in the 'statement 

(-) means the statement does not agree with your feeling 

IV)r Kcvoral other questions, special directions will bo 
iiicludod with the questions. Follow those directions. 

When you have completed the questions, please return this 
booklet to your teacher. 

Thanks for your help. 



You may turn the page and start as soon as you have completed reading the 
^ directions. 
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PLANNING CONSTRUCTION PROJECTS 
••WIIAT DO YOn TIITNK'*? 



I-MLL IN 'J'lIE KOLLOWING INFORMATION: 

Ntiinc ^ Date 

SfliooJ Citv 

Grade (circle one) 8th 9th other (please specify) 
Sox (circle one) Male Female 

Subject taught in this class 

Teacher's name 



ST ART THI=: QUESTION S 

An«3w<^r f^ach statomont by circling the synJaol which best matches your 
actual feelimj: 

(+) means the statement agrees with your feeling 

(?) means your 're not sure how you feel about the thing 
mentioned in the statement 

(-) means the statement does not agree with your feeling 



Circle one for 

each statement 

1. I learned quite a bit about jobs from the + ? 
simulation* 

2. I learned quite a bit about how to work with + 7 
other people from the simulation* 

3. To me the simulation was boring. + 7 - 

4. I would recommend the simulation to my friends + ? - 
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circle one for 
each statement 



S. f would like to qo through more simulations + ? 

lik<'» this one. 

f>. 1 would hav<* rather done something olsc ? 
<iui inq Iho ? imo 1 worked with the simulatrion. 

7. Tho simulation helped to answer some of the + ? 

questions I have about jobs. 

0, The simulation took too long. + p 

9. The simulation was over too soon for me. + ? 

0. Some of the tasks were too complicated or too + ? 

hard for me to do. 

II. The summary helped me to "pull things together." + ? 

l.*i. I enjoyed working with other students during + ? 

the simulation. 

IJ. The activities that I did in the simulation -f ? 

w<»ro cxcritinq to me. 



]4. 



]7, 



I of ton had trouble knowing what to do next 
in the simulation. 



lb. This simulation was a good way of getting 
out of class. 

16. 



Thore were too many tests and forms to fill out + 
with this simulation. 

The px-etest and posttest were difficult for me. + 

18. The simulation preview, activities , and + 
summary fit together vrell. 

19. The preview and the other activities at the + 
beginning helped to prepare me for the simulation. 

20. I liked the way I selected my role(s) in the + 
simulation. 
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For ihr noxt quostiomw cither write in your answers ot check (^) 
I hf rippifipr I- ni>:.wiT?: .is indioai.ed in tho quQy.lUm. 

;m.> WK.ii. wa.s you) vol*- (or toler.) in the inmiJjtion "Pj,uu.imi 
liflucul ion.a I'rogramn"? (Check all that apply) 

Adult Education Teacher Learning Theorist 

Counselor Librarian 



Curriculum Supervisor Nursery School Teacher 



Custodian 



Principal 



Data Systems Analyst Psychologist 

Director of Audio-Visual Recreation Leader 



Services 



Registrar 



Director of Pood Services Secondary School Teacher 

Director of Personnel School Nurse 

Educational Consultant School Secretary 

Educational Researcher Superintendent 

Kiementary School Teacher Other (please specify) 



22. Do you think that you performed well in this role (or rolos)? 



Yes, all of the time 
Yes# most of the time 
No# not usually 
No, not at all 



23. List a few reasons why you liked or did not like your role (or roles) 



24. Would you choose this role (or roles) if you were going to be in the 
simulation again? 



Yes 

Not sure 
Mo 



Describe the one thing which you feel you did best in the simulation 
and the one thing you did least well. Be sure to say why you did well 
or poorly. 

Best T hing Peasons Worst Thing Reasons 
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26» Wliat other roles in the simulation did you find interesting? 
(Check all thai apply.) 



Adult Education Teacher 
Couiisolor 

Curriculum Supervisor 
Custodian 

Data Systems Analyst 
Director of Audio-Visual 
Services 

Director of Food Services 
Director of Personnel 
Educational- Consultant 
Educational Researcher 
Elementary School Teacher 



IiOcirning Theorist 
Librarian 

Nursery School Teacher 
Principal 
Psychologist 
Recreation Loader 
Registrar 

Secondary School Teacher 

School Nurse 

School Secretary 

Superintendent 

Other (please specify) 



27. Why do you find this role (or roles) interesting? If you do not find 
any other roles interesting, can you say why? 



28, Compared to your feelings about the work involved in planning 

educational programs before this simulation, how do you feel now? 

Why? 



I am more interested now 
I am less interested now 
I do not feel any different now 



29. Did you discover any new interests by participating in this simulation? 

Yes, I am now interested in 

No 



30. Name some of the things you liked most about the simulation and some 
of the things you liked least about the simulation. 

Liked Most Liked Least 
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Jl. Write dovm some of your ideas on how the simulation might be made 
better • 



\ 



As soon as you have completed these 
to your teacher. 

Thank you. 



questions r turn in this booklet 
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APPEKDIX D ; 
TEACHER LOG AND 
GENERAL M)DULE EVAEUATDN 
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Module Evaluation 



Teacher Log 
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MODULE HV/UiUATlON 
TEACHER LOG 



ToarlK^r Nairn* 
r i I Y 



(lENH RAL I NSTRUCriQNS 

This instrument package is designed t.o obtain your reactions related to 
\hc simulation module which you are pilot testing as part of the Occupational 
Kxploration Program. Your close association with the module places you in a 
unique positiori to evaluate overall quality, to note problems and to offer 
ruKjqostions for lurther development and/or refinement • Hence , your candid 
appraisal of the module is sought by its developers. Your feedback will give 
direction to the revision process, which will be the next step in developing ^ 
the module. 

The package consists of srveral parts arranged in the order in which they 
should be used. These parts are described briefly below: 



rntroduction 
to Simulation 



frHEN TO COMPLETE 

Upon completion 

of the Introduction 



Estimated 
Time Required 

5-10 minutes 



It- Module Preview 



Upon completion of 
Preview 



5 minutes 



ni- Preparation Phase 



Upon completion of 
the Phase 



5-10 minutes 



IV . l^'irticipation 

Phase (task eval- 
uation) 



As students complete 
each task 



5-10 minutes 
per task 



V- .'nummary Phase 



Upon completion of 
the Phase 



3-5 minutes 
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Part I: INTRODUCTION TO SIMULATION 
SIMUIATION - AN EXCITING WAY TO LEARN 
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Vnrx: I: tNTRODUCTION W StMULATION 
SlMUIiATlON - AN KXCITINC; WAV TO LEARN 



Complote this part after your students have seen the slide presentation im- 
Iroducing the idoa of simulation, have read the booklet which covers the same 
ideas or have used both the slides and booklet together. This part consists 
of sovf>ral brief questions about the introduction to simulation. To respond, 
circle the letter of the phrase that best describes your answer. Several 
(iuestions will require that you supply a short answer. ^ace has also been 
provided for you to write in any comments you have. You are encouraged to do 
so. 



Thanks for your help. 

I. How many total students in your class were introduced to the concept of 
j;lmulation by one or both of the means described above? 

students 



Wow many stud<mts used: (count each student only once) 



The booklet only 
The slides only 

The slides first and then the booklet 
The booklet first and then the slides 
Other, please specify 



i. Were the students able to understand concepts presented in the materials? 

a. Yos, most of the time Comments 

b. Somewhat 

c. No, not much of the time 

4. Vfeis the vocabulary consistent with the maturation'*^! level of the students? 

a. YeS;. mc^st of it Comments 

b. Some of it 

c. No, not much of it 

5. How would you rate the quality of the illustrations used on the slides and 
in the booklet? (Answer both parts of the question.) 



a. 
b. 
c. 
d. 

0. 



Slides 

Very Good 
Good 
Average 
Poor 

Very Poor 



Booklet 

a. Very Good 

b. Good 

c. Ave f age 

d. Poor 

e. Very Poor 



Comments 
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6. 



Overall, how would you rate the toclinical quality (appea'ance, oase of use, 
etc.) of the slides and booklet? (Answer both parts of the question.) 



Sl ides Booklet Comment s 

a. Ver^' Good a. Very Good 

b. Good b. Good 

c. Average c. Average 

d. Poor d. Poor 

«. Very Poor e. Very Poor 

7- Ovtsrall, do you feel that this introduction was stimulating to students? 

a. Yes, very much Comments 

b . Somewhat 

c. No, not much 

B. in what order would you recommend the use of the slides and the booklet? 
(Choose only one . ) 

a . Use both in any order Comments 

b. Use both v;ith the booklet first 

c. Use both with the slides first 

d. Use the booklet only 

e. Use the slides only 

f . None of the above 

9. Would you recommend the use of the slides anchor the booklet to other teachers? 
(Answer both parts of the question.) 

Slides Booklet Comments 

a. Yes, with a. Yes, with minor 
minor modi- modification 

f ication 

b. Yes, with b. Yes, with major 
major modi- modification 

f ication 

c. No, I would not c. No, I would not 
recommend it recommend it 



Please write in any other comments/suggestions you might have in the space 
below. (If extra space is required, use the back of this page.) 
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Part II: MODULE PREVIEW 
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PACT III - PREPARATION PHASE 
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Complete this part after your students have finished the preparation phase 
of the simulation module • Questions here relate to the materials used to pre- 
pare students for participating in the simulation and the actual process of 
getting students into roles. 

To respond, circle the letter of the choice that best describes your answer 
or how you feel. Some questions will require that you either check (v^ an 
answer or write in a short response. Space has also been provided for you to 
write any comments or suggestions you might have. You are encouraged to do so. 

Thanks for your help. 

MATERIALS 

1* In the following chart: describe or ncune the form of material used (e.g., 
slide tapes, booklets, combination of forms, etc.); specify how many stu- 
dents used the form counting each student oniy«.once; rate the technical 
quality of the form; and rate its overall educational quality or worth. 

Ratings are indicated by placing a check (v/) in the appropriate box in 
the applicable cell. You are encr":-.aged to pladfe" comments and/or descrip- 
tions of problems you encountered in the large space provided in each box. 



1 N«i«« of Toxvk 


t.'o. of 1 
•tudanta 


1 RAto the technical quality 

Uppcitranca^ of uaa* 
1 ate.) or t>M totm 


l.^ic tha ovarall lAl-watiofal 
(M^Uty of thr Tom 




Hlqh 


























COHMHtt 








CctvRfnta 


















1 1 |cc-.r.t. 


1 


















1 j |canMnta 


1 




c<v.fi*nt» 









2. Were the students able to understand the concepts presented in the materials? 

a. Yes, most of the time Comments 
b* Somewhat. 

c* No, not much of the time 
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was tho vocabularly consistent with the maturational level of the students? 

of it Conments 
l>. i>ome of it 

c. No^ not much of it 

.To what extent was the preparation phase integrated with (i.e.', how well 
did It fit together with ) the Module l^review? 

a- Very well, integrated Conments 

b. Will integrated 

c» Somewhat integrated 

'*oorly integrated 
V. Very |x)orly integrated 



ROLE SELECTION PROCESS 

Did the initial role descriptions provide students with enough information 
for selecting roles? 

a- Yes, the Lnformation was adequate Comments 
I)* Somowhat 

c. No, the information was inadequate 

If schematic devices (e.g., schedule cards were available to help select 
roles, did studorits understand how to use them? 

a. Yes, with little or no help Congnents 

Yes, with some help 
c. Yes, with a great deal of help 
d* No 

c. Not applicable 

Were the students able to independently select themselves into roles? 

a- Yes, with little difficulty Comments 
Yes, with some difficulty 
No, some teacher assistance 
Wfi« necessary 

<!• No, extensive? toacijer assis- 
tance or direction was 
necessary 

If you had to help tiie students select roles, please describe the nature of 
^.i'^t assistance (e.g., asked students to draw lots when several wanted ♦-he 
same role; explained use of schematic device; etc.) in the space below. 
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Overall, wai; the role selection process described in the module an effec- 
tive way of getting students into roles? 

a. Ye:5, it wan effective Comments 

b. IJomewhat effective 

c. No, it was Ineffective 

30. Can you suggest other ways ia which this process could occur? 
a. Ycr., I would suggest 



b. No, the process was effective 



t*loc*.-.c write in any other comments/suggestions you might have in the space below. 



ERIC 



PART IV - PARTICIPATION PHASE 
TASK EVALUATION 
SKILLS PACKLT 



ERIC 



o 
<c 

—J 

<c 

CO 

-a: 



csel 
<C 
a. 



«) \ w 
4J O -* C 












g 






p. 




•H Id 


o o 






hi ^ 










•0 c 


u >* 






c 










1/1 Ji! 






4 Ul 


C-t o 




4J Its 






4J 


#£ •H 










O -C 


0* ? 




<d u 






« Id 


« o 






'/) 






O <d 






M c> 




•H O 






a 


u o« 






Id 










M 


r 










c c 


2 


> 




o 




u 


3 Xi 


a: 


cu 


01 


^ < 


CO 








M *H 


to o 


o 


O Q> 


S tJ 


a 


>i 4J 




n 








(0 ^ 

<d 5t 


s 


a> 












CO €> 






•H Ul 




c 


(0 <d 






10 0) 
















> 




• o 


<d 


05 


c CJ 




ig • 


E O 

3 0 








o 


>• :j 


O CO 


>, 


^ •5 


O 3C 


Ul 


^ i 




c 


CO 


u cn 


o 




nJ O 


•H 


55 




4J 






Ul 






Q> 




o o: 


tjv 




u 




^ 3 


o*Ej 


1/1 




t; 




«> Ul 








0) p: 


Q 






\ 




•p 


•o 




cn 


c 








O <M 


Id iJ 




u o 










0) 


O 0) 




c 


'A 0) 


xi MX 


o 


Id 


Ul (-< 


•H 




-1 < 








:j 


cn 


•H 


o c 


Ul Q« 


u 


x: o 


c o 


u 


01 -H 


o a: 


Ul 




•H Q« 




J? 




*o 




Ul < 






0) 


E 


Ot U 


3 u 


o 


•H 


u«:c 




0) 4> 




JO 




0) 


o 






u 


^ s 




a 



o 



X 

p 

o 
p 

a 

& 
o 
u 

o 
>• 

a 
«-i 

X 

o 
p 

•o 

> 
o 



c 
o 

P 

NS 

> 
CO 

p 



o 

§ 



o 

C A* 



4* 4J 



3.1 



1: n 



^ «: •« t» 

^ ». i» •: c 

X 0 <: e w ^ 

o ^ o v • 

^4 O t- f.» C» «5 

v« -J :• n ^ 

> I .: ^ *: « It 

u » u -c • « _ 

c cr o «» o t>* • 

o. 'J o c • fi 4: 
w i« ****** ^ ** 



Is 

w *« 

23 



O M 4 

*> t» « O 

fl p ** .4 

h « o c w 

e* < • o • 

m v > u%* jc 

c%4 u B u ^ 



I Oi u o 



k» m V 

u ^ c m 

m V o 

-4 -4 «« 
%t \* o 

0.0 * 

fc* M 3 -tt 

Q,m >.e 

a o 9 • • 

< C C M « 



.>• 4 O V 0« 

1*4 JS ^ ^ C 

<-« U C €£ 4 

9 o r- JC 

fi 1« C p 

i O J< o 

vt •>« >• 

g« M «* «4 

• Q *« >3 •« 

• M n j» 



Q. O. 
O* B 
U « O 

o u r 
up J 

E U ( 

• -H O < 
»4 *« %4 4 



• 



>. 3 C 

C «* « • 

V ^ U M *« 

• H*P<*4 -40'»« 



m 9 e .V P 

S o c 

o ^ o >• *« C 



55 



IL 



^{ o ^ 



.1 



J.? 



IL 



5 



I 



1 
>4 



I 



2. 



o 



OS 



BEST COPY AVAILABLE 




V O V 

§u ^ 
o. o. 
Qt a 
o m p 
9 u r* 
urn M 
cum 
ffl ^ o « 




fl> « C 2 

-1 IJ r , «5 - 

•« ill c: ^ c H 

u w o w •« II 



ERIC 



1a5 



CO 




ERIC 



1^^ 



(0 

c 
o 

(D 

to 

Ui 

c 



U * 



J2 
+J 

^ O 
•H 
Ui Ui 

§-H • *. 
> M 
0 0 0 
V4 X W 

o 

0 
Xi 

Ui 

c 
o 

■p 

0 

CP V 
CP «3 

Ui 4J 



o 

V4 

Qi Ui 
■P 
0 

0 o 

(d 04 
o 

•r! 

H X 



4-> 

CP 



■p 0 

0 > 

X 0 

O .H 
(d 

o o 

•H 

■p (d 

:J :3 w 

u -P -P 

U O 



0 Ui 



0 

■p 



*j G :j 

CI o 

0 *j >i 
■p 

a -H o 



Too 

easy 




Just 
right 




Too 
hard 











0 


0 






'O 




U) 


0 






0 






c 




0 






■p 


c 






0 




c 






•H 


iJ 




■P 






0 








0 




0 








c 






•H 




(0 


Cn 




•H 


c 


to 


J2 


•H 


■p 


■P 


'O 


c 




•H 


0 


0 


> 




■P 


0 


:3 








5 




to 











0 




s 





















to 
■p 
c 

0 CP 

o :5 
■p 



■p 
0 

c to ^ 

to J3 (d 



tjt o) :J 4J Q< 



O 4J 
iH 0 

^ to O4 



PART V: SUMMARY PHASE 



ERIC 



PART V: SUMMARY PHASE 



Coinplete this part when your students complete the "Sumnary" phase of the 
Tc^l^s .2/^^"°''^' """P^^^^l^ letter beside the phrase that best de- 
uroiidL T ^""^T or supply the requested infonnation. Space has also been 
provided for you to write in any conunents/suggestions you may have. 

Theinks for your help. 

1. HOW effective was the "Summary" phaso in providing a reasonable culimina- 
t-ion to the simulation experience? 

a. Very effective Comments 
Somewhat effective 

Not effective 

2. was the;'Summary«» phase well integrated with the immediateLy pr^cedina 
activities or tasks? 

^* Ccnunents 

b. SoRiewhat ' ^ 
c* No 

3. Did you have to modify or expand upon the Summary"? 
a. Yes, I did tho following 



No 



natin"r'''r f*= "--^.^ry" phase in helping students learn about occu- 
.f. pational roles performed by others in the simulation? 
}' 

a. Very effective Cor.u-.ents 

b. Somewhat effective 

c. Not effective 

5. HOW effective was the "Summary" phase in helping students learn about tools, 
processes and working conditions associated with that part of the world of 
work simulated in the module? wuxxu 



a. Very effective 

b. Somewhat effective 

c. Not effective 



Comments 



o 1:^,3 



6. How useful do you feel the "Summary*' phase would be in helping students 
identify and select alternatives for further action related to other 
occupational exploration activities? 



a, Vory useful Commonts 
b* Somewhat useful 
c. Not useful 



Please write in any other comments/suqgestions that you might have in the space 
be low ♦ 
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GENERAL MODULE EVALUATION 
TEACHER FORM 



INSTRUCTIONS 

This questionnaire should be filled out as soon as possible after 
the pilot test of this module has been completed, i^e,, after the 
posttests have been given. 

The questionnaire is divided into several sections. The first 
section deals with general background characteristics of students and 
teachers. This information will be used solely for the purpose of 
describing the students and teachers who participated in the pilot test 
of this module. Subsequent sections will deai with implementational 
problems, your perceptions of the quality of the materials, etc. 

Fill in the information requested at the top of the questionnaire 
and then answer each question by circling the letter in front of the 
j)hrasc that best describes your answer, unless given other specific 
directions in the question. Space has also been provided for you to 
write in any comments/suggestions you might have. You are encouraged 
to do so. 

THANKS FOR YOUR HELP. 
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GENERAL MODULE EVALUATION 
TEACHER FORM 



Module Name ^*l*^ing Educational Programs 



Teacher Neune 



School 
City 



TEACHER BACKGROUND 

1. What is your sex? 

a. Male 

b. Female 

2. Including this year, approximately how many years of teaching 
experience do you have? 

a. This is my first year 

b. 2-4 years 

c. 4-6 years 

d. 6-8 years 

e. 8 or more years 

3. In what kind of group setting (e.g., English classroom, math 
classroom, students from study hall, students from a guidance 
group, etc.) and at what grade level did you introduce this 
simulation? 

a. Group Setting (please specify) 

b. Grade Level (please specify) 

4. Have you had any previous experience with simulation as an 
instruc'::ional technique? 

a. Yes, as a teacher 

b. Yes, as an observer 

c. Yes, as a participant 

d. No 
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5^ If you answered yes to question 4, briefly describe the nature 
and extent of your previous experiences with simulation. If 
your response to question 4 was 'No' please proceed to question 6. 

a. My previous experiences with simulation include 



6. Which of the following statements best describes your reasons for 
participating in the pilot test of this simulation module? 

a. To try out new ways of organizing instruction for students 

b. Interest in Career Education 

c. Thought material was of value for students 

d. General interest or curiosity 

e. I was requested to participate 

f. Other, or some combination of the above (please specify) 



STUDENT BACKGRCXJND 

7. How many students participated in the total simulation? (Include 
only those students who were involved in the module and received 
both the pre and posttests) . 

Students Participating 

8. Of the students in question 7, how many were male and how many 
were female? 

Males Females 

9. How were students selected to participate in the simulation? 

a. Most of the students were volunteers from the class 

b. The class, rather than the students, was volunteered 

c. Student volunteers from a study hall 

d. Other, please specify ^ 
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10* If you had volunteer students participating in the simulation 
which of the following reasons best describes your perception 
of why they participated. If you did not have any volunteer 
students please proceed to Question 11. 

a. Interest in trying something new 

b. Interest in particular area simulated 

c. Interest in careers 

d. Interest in just getting out of class or study hall 

e. Other, or some combination of the above (please specify) 



f. I can't really guess at the reason (s) 

.11* 'Indicate any special characteristics of this class, e.g., many 
slow readers in class; many students with exceptionally good 
verbal skills; etc., which may bias the results of the pilot 
test of this module. Also describe how you feel the results 
will be biased by these characteristics. 

a. Characteristics Biases Produced 



b. No special characteristics 



SEQUENCING OF MATERIALS 

12. In general, how well did the transition from phase to phase of 
the irodule proceed? 

a. Very we 13 Comments 

b. Well 

c. About average 

d. Poorly 

e. Very poorly 

13. Are there any additions, deletions or changes in the order of 
module parts that you feel should be made? 

a. Yes, mcike the following changes 
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b. No changes are necessstry 



ADEQUACY OF MATERIALS 



14. In general, were the directions in the nodule clear enough for 
students to understand what was expected of them? 

a. Yes Coments 

b. Somewhat 
c« No 

15. In general, was the vocabulary of the module consistent with the 
maturational level of the students in the siraulation? 

a* Yes 

b. Somewhat 

c. No 

16. Do you feel that the knowledge (What do you know?) and the atti- 
tude (What do you like?) tests were adequate measures of the 
•material contained in the module? (Answer both parts of the 
question.) 

Knowledge Test Comments Attitude 'rest Comments 

a. Yes a. Yes 

b. Somewhat b. Somewhat 

c. No c. No 

17. To what extent was the knowledge test difficult for students? 

a. Very difficult Comments 

b. Difficult 

c. About average 

d. Easy 

e. Very easy 



IMPLEMENTATION OF THE MODULE 

18. How well did the in-service training prepare you to work with^ 
this module? 

a. Very well Comments 

b. Well 

c. Somewhat 

d. Poorly 

e. Very poorly 
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19. Did the in-service training provide you with a general understanding 
of your role in the module implementation? 

a. Yes Comments 
b • Somewhat 
c. No 

20. Vhile working with this module, did you have to allot (or spend) more 
time than you normally would for preparation (exclude the time spent 
in in-service training)? 

a. Yes, specify additional time Coimnents 
in hours 

b. Some extra time was necessary 

c. No extra time was necessary 

21. How sizeable was the job of managing/coordinating this simulation 
module for you? 

a. Very sizeable Coianents 

b. • About average 
c* Not sizeable 

STUDENT PARTICIPATION AND LEARNING 



22. Did your students experience problems with the reading level of this 
module? 

a* Yes COimnents 

b . Somewhat 

c. No 

23. To what extent do you feel students were receptive (interested in, 
excited by) to simulation as a way of learning? 

a. Vexry receptive • Comnents 

b. Receptive 

c. About average 

d . Non- receptive 

e. Very non- receptive 

24. To what extent do you feel that students were receptive (interested 
in, excited by) to the content of this particular module? 

a. Very receptive Comments 

b . Receptive 

c. About average 

d. Non-receptive 

e. Very non-receptive 
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25. Was there any change in student interest or motivation as they 
progressed through the module? 

a. Yes, interest changed as follows 

b. Somewhat ~ 

c. No 

26. Do you feel that this module reinforced or helped to build the 
student's ability to make decisions? 

a. Yes Comments 

b. Somewhat 

c. No 

d. Don't know 

27. In your judgment, how much did students learn about the process 
of simulation and about the content of the module? {Answer both 
parts of the question) 

Simulation Process Coitinents Module Content Comments 



a. 


Very Much 


a. 


Very Much 


b. 


Much 


b. 


Much 


c. 


An average amount 


c. 


An average amount 


d. 


Little 


d. 


Little 


e. 


Very little 


e. 


Very little 



OVERALL PERCE'r-TIONS AND RECOMMENDATIONS 

28. In general was this module 

a. Kxciting to students. Comments 

b. About average for 
students. 

c. Boring to students. 

29. In general did this module change the working relationships 
(personal interactions) between you and participating students? 

a. Yes, relationship changed as follows 



b. Somewhat 

c. No 
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30. Are there any students or groups of students (e.g., some students 
may havo difficulty working in small self-directed groups) that 
you feel would have difficulty participating in simulated types 
of experiences? 

a. Yes, (please specify) 



b. No 

31. For what grades would you consider this module to be appropriate? 
(circle as many as apply) . 

a. 10th or higher Comments 

b. 9th 

c. 8th 

d. 7th or lower 

c. Other, please specify 

32. Overall, how would you rate the quality of this module? 

a. Very good Comments 

b. Good 

c . Average 

d. Poor 

e. Very Poor 

33. if possible, would you use this module with students again? 

a. Yes, with no modification Comments 

b. Yes, with minor modifications 

c. Yes, with major modifications 

d. No 

34. Would you recommend this module to other teachers? 

^* Yes Comments 
b. No 
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COMMENTS AND/OR SUGGESTIONS FOR REVISI ON 

Add as many comments and/or sucjqostions for revision oC the module^ <is you 
miqhfc have. 



THANK YOU. 
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